Papers with multilingual pre-training
Multilingual Neural Machine Translation (2020.coling-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will cover the latest advances in NMT to enhance low-resource translation. |
| Approach: | They will cover the latest advances in NMT approaches that leverage multilingualism . they will focus on topics such as language divergence, transfer learning and pivoting . |
| Outcome: | This tutorial will cover the latest advances in NMT to enhance low-resource translation models. |
On-the-fly Cross-lingual Masking for Multilingual Pre-training (2023.acl-long)
Copied to clipboard
| Challenge: | In multilingual pre-training, multilingual models only learn cross-linguality implicitly from isomorphic spaces formed by overlapping different language spaces due to the lack of explicit cross-linguistic forward pass. |
| Approach: | They propose a dynamic token-wise masking scheme for multilingual pre-training that uses a special token [C]x to replace a random token in the input sentence. |
| Outcome: | The proposed model improves the performance of UNMT models on De, Ro, Ne En. |
SingaKids: A Multilingual Multimodal Dialogic Tutor for Language Learning (2025.acl-industry)
Copied to clipboard
Zhengyuan Liu, Geyu Lin, Hui Li Tan, Huayun Zhang, Yanfeng Lu, Xiaoxue Gao, Stella Xin Yin, Sun He, Hock Huan Goh, Lung Hsiang Wong, Nancy F. Chen
| Challenge: | Empirical studies show that SingaKids provides effective dialogic teaching, benefiting learners at different performance levels. |
| Approach: | They propose a dialogic tutor designed to facilitate language learning through picture description tasks. |
| Outcome: | Empirical studies show that SingaKids provides effective dialogic teaching, benefiting learners at different performance levels. |
SPEED++: A Multilingual Event Extraction Framework for Epidemic Prediction and Preparedness (2024.emnlp-main)
Copied to clipboard
Tanmay Parekh, Jeffrey Kwan, Jiarui Yu, Sparsh Johri, Hyosang Ahn, Sreya Muppalla, Kai-Wei Chang, Wei Wang, Nanyun Peng
| Challenge: | Prior studies focused on English posts to provide early warnings for epidemic prediction, but these work focused on non-English posts. |
| Approach: | They propose a multilingual event extraction framework for extracting epidemic event information for any disease and language using 5.1K tweets in four languages. |
| Outcome: | The proposed framework can provide epidemic warnings for COVID-19 in its earliest stages in Dec 2019 (3 weeks before global discussions) and aggregate community epidemic discussions like symptoms and cure measures, aiding misinformation detection and public attention monitoring. |
Asymmetric Conflict and Synergy in Post-training for LLM-based Multilingual Machine Translation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing work in LLM-based MMT typically mitigates the Curse of Multilinguality . asymmetric phenomenon in linguistic conflicts and synergy varies in different translation directions . |
| Approach: | They propose a direction-aware training approach to address asymmetry in linguistic conflicts and synergy . they propose X-ALMA-13B-Pretrain with multilingual pre-training to achieve comparable performance . |
| Outcome: | The proposed method achieves comparable performance to X-ALMA-13B-Pretrain (only SFT) with fewer pretraining tokens and 17B parameters. |